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ABSTRACT 


Technological advances and increased operational 


challenges have led to the introduction of automated agents 








into military teams. Although these new combined teams have 

















many advantages, it is possible that the interactions 
between members of these new human —- automation teams may 
adversely impact mission accomplishment. This study 





investigates the similarities and differences between human 
- human teams and human —- automation teams with respect to 


team communications, efficacy, and trust. Thirty-six 





participants were formed into twelve thr person teams. A 





confederat served as the fourth member for all twelve 








teams. In the human - human team condition, the confederate 


was present in the same room as the other three team 





members. In the human —- automation team condition, the 


confederate was located in a separate room and the other 

















thr team members were told that their fourth team member 
was an automated intelligent agent. All teams played a 
computer-based team firefighting game (C3Fire). The order 


of presentation of the two trials (human - human vs. human - 


automation) was counterbalanced. The results of this study 








indicate there is a significant difference in the nature of 





the communication between these two types of teams. 





Additionally, the presence of an automated agent changes the 
nature of trust and team efficacy. These findings 


demonstrate the need to consider the unintended impact of 





including automated agents on team dynamics in military 


environments and other complex and dynamic systems. 
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EXECUTIVE SUMMARY 


The technological advancements in the latter part of 
the last century led to an unprecedented proliferation of 


complex systems. Automated systems have become ubiquitous 





to the point that they are now being introduced into teams 


that were previously composed exclusively of humans. While 








the addition of automated agents may increase some aspects 





of mission ffectiveness, the nature of the interactions 
between human -—- human teams and these new human - automation 


teams needs to be studied carefully. Technology is often 





thought of as the answer to many problems associated with 





increasing operational demands in a resource-constrained 





environment. However, the new challenges that emerge when 
automated systems are introduced are often not fully 
considered. These challenges can be mere inconveniences in 
some cases but catastrophic in other cases. This study 
investigates the differences in the interactions of human - 
human teams and human-automation teams through an analysis 


of team communications, efficacy, and trust. 





Thirty-six participants were formed into twelve thr 








person teams. A confederate served as the fourth member for 





all twelve teams. In the human —- human team condition, the 





confederate was present in the same room as the other three 





team members. In the human — automation team condition, the 


confederate was located in a separate room and the other 














thr team members were told that their fourth team member 
was an automated intelligent agent. All teams played a 
computer-based team firefighting game (C3Fire). The order 





of presentation of the two trials (human —- human vs. human - 


automation) was counterbalanced. 


xX1il 


Results of this study reveal compelling evidence that 











the addition of an automated agent to a human team changes 
the nature of communications amongst team members, impacts 
team efficacy, and alters the task distribution of the 
team. Statistical analysis showed that even though the 
total number of communications amongst both group types was 
equivalent, the type of communication was different. In 
human - human teams, information was more likely to be 


shared to aid in decision-making, 


teams, 
nature. 


affect 


efficacy, 





directiv 





while on human-automation 


in 





communications wer 


While th 





presenc 


overall 


mor 


likely to b 


of an automated agent did not 


team performance, it 


and task distribution of the team. 





did affect trust, 


Statistical 


analysis 


indicated that human 





team members 


had great 


Cer 


confidence 
automated 
members 


affected 


suggest that there 


interactio 





in another human team member 


agent. 


trusted the automated agent less, 


task distribution. The 


is are 


ns of human —- human teams 


results 
important differences 


compared to 





compared to the 


Analysis also showed that the human team 


which directly 


of this study 





in the 


human- 


au 


tomation teams and demonstrate the need to consider the 


unintended impact of including automated agents as team 





members in military environments and other complex and 


dynamic systems. 
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I. INTRODUCTION 


A. PROBLEM STATEMENT 





The last century led to an unprecedented level of 
technological development and implementation. This explosion 
in the use of automation requires humans and machines to 


work together to effectively perform required tasks. The 





ability of a combined team to perform effectively in the 


face of difficult problem solving situations requires proper 





coordination, collaboration, integration and trust between 
all team members. For the military, these fundamental 


changes in technology and automation have altered the very 





nature of warfare. No longer do riflemen stand in a field, 
aim at the opponent and shoot at each other. Now, 
warfighting methods include computers, unmanned aerial 
vehicles (UAVS) and robotic agents. This fundamental change 
in the nature of warfare has transformed the normal team 
composition from teams consisting of entirely human members 


to teams consisting of humans and automated agents. This 





thesis seeks to investigate the possible differences between 





the nature in which humans interact with other humans as 


compared to their interaction with automated agents. 


As the nature of warfare has changed, the resulting 


task complexity has led to an increased use of more 





sophisticated technological systems. These advances in 
technology have introduced intelligent agents (IAs) into 
military teams. While there is no universal definition for 





TAs, Wooldridge defines an IA as one capable of flexible 





autonomous action in order to meet its design objectives 





(Richards, 2003). In this definition, flexibility means the 








IA is able to perceive and react to the environment, is able 





to take initiative to satisfy its design objectives, and is 





capable of interacting effectively with other agents, 


including human beings. 


These advances in automation can significantly reduce 





manning levels, while increasing mission effectiveness, but 





they are not a panacea. One approach used in the past has 





been to automate the functions that are easiest to design 


and leave the rest to the human operators. This approach may 





not reduce the number of tasks assigned to the human, but 
merely change their nature. According to Sheridan (2002), 


this type of design may lead to ineffective automation 





because the remaining human tasks become more complex and 





less understood, resulting in the overall degradation of 


system performance. 


While there is a large amount of research regarding 
team performance (Salas and Fiore, 2004; Patrashkova- 


Voldoska, McComb, Green, & Compton, 2003; Cook, Salas, 





Cannon-Bowers & Stout, 2000), som researchers suggest 
little is known about what is important in an effective 


human-intelligent agent team. Fiore, Jentsch, Becerra-— 





Fernandez, Salas, & Finklestein (2005) argue that it is 


unclear what information human-agent teams use in order to 





ffectively perform a given task, and how the presence of 





non-human team members alters what have been the traditional 








requirements of effective teams. Additionally the degree of 
automation required on each team for a defined task is 


uncertain. Other researchers have suggested that effective 








human-automated teams begin with a system design that 


incorporates two fundamental characteristics: observability 
and directability (Christoffersen & Woods, 2002). The 


design phase is followed by incorporating effective feedback 








mechanisms to share all information amongst team members. 


Researchers have discovered that providing the right 


degree of automation for the proper function of a task can 





optimize overall team performance (Wright, 2005), but 








automation design and implementation is not an exact 








science. Since the military has limited resources, 
integrating automated agents into traditional human —- only 
military teams remains challenging. This effort will 





require a proper understanding of how to effectively design 





and implement these teams. Additionally, human team members 
will require a proper understanding of the intelligent 


agents in order to maximize mission effectiveness. Because 








warfighting will become an activity in which humans and 


intelligent agents are even more interdependent than they 





are at present, it is critical that we understand the nature 








of interactions on these newly formed teams. 


B. OBJECTIVES 


The objective of this research effort is to compare the 








nature of the interactions between members of a purely human 
team and a team consisting of human members and an 
intelligent automated agent. The specific goals of this 


effort include: 


e To analyze the difference in frequency of 
communications between human teams and human- 
automation teams; 





e To analyze the nature of the communication to 
determine the differences in interaction between 











= 


members of a purely human team as compared to a 
team consisting of human and intelligent agents; 


e To assess differences in team efficacy between 
human teams and human-automated teams 


e To evaluate the results with respect to Human 
Systems Integration (HSI) and potential effects 
on the field of HSI 

















Cc. RESEARCH QUESTIONS 


The specific research questions addressed in this study 














include: 

e Are there differences in the nature of 
communication between human teams and teams that 
combine humans and an intelligent automated 
agent? 

e If so, how do these differences affect team 
performance and team efficacy? 

e What elements should designers consider in 
building future systems that partner humans with 
intelligent automated agents? 

D. HUMAN SYSTEMS INTEGRATION (HSI) 





HST is less a process and more a field or discipline 
that recognizes that, in any complex system, humans play a 


vital role. According to Booher (2003, p.4), HSI is 





“primarily a technical and managerial concept, with specific 


emphasis on methods and technologies that can be utilized to 








apply the HSI concept to systems integration”. According to 








the U.S. Navy, HSI is a multidisciplinary field of study 


consisting of eight basic areas or domains of study: 


e Manpower 
° Training 
° Personnel 


° Human Factors Engineering 








e System Safety 

e Personal Survivability 
° Health Hazards 

e Habitability 





This thesis will focus on three of the eight HSI 
domains: personnel, training, and human factors engineering. 
The personnel domain focuses on the knowledge, skills, and 
abilities of the people that are required to operate, 
maintain, and support a system. This domain involves both 
the physical and cognitive attributes possessed by the 


individual (Archer, Headley, & Allender, 2003). The U.S. 








Department of Defense, and specifically the U.S. Navy, has 


been reducing manning levels significantly in order to 





reduce manpower costs, often utilizing technology and 








automation in an attempt to maintain mission effectiveness. 











Determining the correct personnel to fill the billets that 





require humans to interact with automation has proven to be 





challenging becaus th skill sets required for these 








billets are often unknown. This thesis will provide some 





insight into the nature of interaction between combined 





military teams, which should aid in determining critical 


factors when assigning these billets. 


The training domain focuses on ensuring that the 


training requirements and programs will allow the personnel 








to properly operate, maintain and support the system (Archer 





et al., 2003). This domain will certainly need to change as 
the reliance on automation increases. In order for human- 
agent teams to b ffectively utilized, the conditions under 











which training needs to be tailored and delivered requires 





5 


further investigation (Fiore et al., 2005). The results of 


this thesis will provide insight into the nature of training 





requirements for future military teams. 


Human factors engineering (HFE) is “the integration of 
human characteristics, into systems definition, design, 


development, and evaluation to optimize human-machine 








performance under operational conditions” (Lockett & Powers, 





2003, p.463). Some of these characteristics, such as 
physical size of the human population may seem rather 
obvious, but other factors such as human cognition and trust 
in automation are not quite so apparent. This thesis will 


explore some of these human characteristics in an effort to 





gain a better understanding of how they relate to the 





ffectiveness of future combined military teams. 


E. THESIS ORGANIZATION 











In Chapter , literature regarding team performance, 








communication, trust and team efficacy is discussed. Chapter 














outlines the methods used to conduct the experiment and 





describes C3fire, a microworld used to assess team 





performance and evaluate communication metrics. Chapters IV 





and V present the results of the experiment, a discussion of 
these results, conclusions, and follow-on research 


recommendations. 


II. LITERATURE REVIEW 


This literature review is divided into four sections. 





The first section focuses on team performance. The second 


section discusses communication techniques and communication 





frequency among members of a team. The third section 


examines the factors involved in trust among team members. 





The final section discusses the factors involved with team 


efficacy. 


A. TEAM PERFORMANCE 


A team iS a mature grouping of individuals that 


generates synergistic effects through a coordination of its 








members’ efforts (Robbins & Judge, 2007). A team may also 
be defined as “collectives who exist to perform 
organizationally relevant tasks, share one or more common 





goals, interact socially, exhibit task interdependencies, 


maintain and manage boundaries, and are embedded in an 





organizational context that sets boundaries, constrains the 





team, and influences exchanges with other units in the 





broader entity” (Mathieu, Maynard, Rapp, & Gilson, 2008, p. 
411). The primary functions of teams, and a common reason 


for their formation, are to share information, develop 





strategies, make decisions, and accomplish tasks. If the 
difficulty of a task is too great for an individual’s 


abilities, cognitive or otherwise, then the formation of a 





team may be necessary. But, according to Salas and Fiore 





(2004), it is a common misconception that the formation and 








implementation of a team will lead to success. Research 








indicates that in order for a team to b ffective, it must 


develop a shared cognitive capability referred to as team 
cognition (Burnett, 2006). The end result of teamwork can 


be considered the result of the collaboration of individual 





cognition, behaviors and attitudes and it is important to 





understand the factors that affect team performance and 


ffectiveness (Salas & Fiore, 2004). 





Salas and Fiore (2004) define team cognition as the 





cognitive development that results when members in a team 
are engaged in complex and dynamic task accomplishment. 
Cannon-Bowers & Salas (2001) suggest that shared cognition 
can be a useful concept in describing team performance in 


several ways. First, shared cognition has value to explain 








how members of a team interact with one another. When 


highly effective teams are observed, they often coordinate 





their behavior without the need for verbal communication. 











In thes ffective teams, members have similar or compatible 
knowledge that guides their coordinated behavior. 


Additionally, the concept of shared cognition may have the 











potential to predict the likely effectiveness of the team. 
This predictive power may be able to identify potential 


problems and provide some insight into possible solutions. 





When team members have similar thoughts, knowledge, and 





experiences and utilize these attributes to coordinate their 
efforts, this shared cognition can be a good predictor of 


team effectiveness (Cannon-Bowers & Salas, 2001). 








In completing assigned tasks, team members develop 





mental models that directly affect decision-making processes 





and team performance. In ssence, mental models are 





organized knowledge structures that allow individuals to 





interact with the environment around them, explain and 


predict the behavior in their environment, and recognize and 


remember relationships in order to predict what is likely to 





occur next (Mathieu et al., 2000). Other researchers state 





“mental models are the mechanisms whereby humans are able to 





generat descriptions of system purpose and form, 


explanations of system functioning and observed system 





states, and predictions (or expectations) of future system 








states” (Rouse, Cannon-Bowers & Salas, 1992, p. 1300). 





Figure 1 describes these thr functions of mental models: 





description, explanation, and prediction. 


Purpose —> Why ASystem Exists 


Describing 


Function —» How A System Operates 
Explaining 
State —> What A System Is Doing 
Predicting 
Form —> What A System Looks Like 


Figure 1. Nature of Mental Models (From: The Role of Mental 
Models in Team Performance in Complex Systems, 1992, p. 
1300) 


The description function involves knowledge of why a 
system exists and what it looks like. The explanation 
function describes how a system operates and interprets what 
the system is currently doing. The prediction function 


enables formation of expectations about future events 





involving the system. Of course, it stands to reason that 





the explanatory and predictive functions interact; the 
explanation of how a system functions has an impact on 


predicting future states. 


Rouse, Cannon-Bowers and Salas (1992) assert that 








technology has allowed th development of increasingly 








sophisticated systems throughout all walks of life. This 














sophistication increases the complexity of activities for 
operators, maintainers, and managers of these systems. 


Furthermore, automation serves an increasingly fundamental 





role. The nature of the decision making process in these 





complex systems can be turbulent and often requires the 


coordinated efforts of multiple team members, whether humans 





or automation. Team dynamics, performance, and ability to 





integrat ffectively become increasingly important for the 


success of these teams. 


The following list from Rouse, Cannon-Bowers, and Salas 





(1992) indicates some of the characteristics of decision 





making in these complex environments that present themselves 
as challenges in highly sophisticated human and automated 
systems. 


e The environment is highly dynamic and sometimes 
hostile; as a result, situations and rules can 
change quickly and risks abound. 
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e Goals vary in time and not infrequently conflict; 
shifting objectives and priorities, as well as the 
impossibility of satisfying all goals, result in 
no “best” decision and perhaps many “acceptable” 
decisions. 














e Information is typically incomplete, uncertain, 
and ambiguous; consequently, it may not be clear 
what can or cannot be assumed, as well as what is 
known or unknown. 








e Typically multiperson teams are involved; thus, 
overall performance depends on more than just 
individual performance. 





e Members of teams have differing roles and 
responsibilities; consequently communication and 
coordination are central issues. 





e Decision making is embedded in an organizational 
context; therefore, team performance must be 
consistent with objectives and constraints 
external to the team. 























Some research suggests that not all team members may 


share the same mental model. Klimoski and Mohammed (1994, 





p. 432) state “there can be (and probably would be) multiple 


mental models co-existing among team members at a given 





point in time” that would include models of task and 
technology, response routines, and processes of team work. 


But it has been hypothesized that the greater the overlap or 





commonality among team member’s mental models, the greater 








the likelihood that members will predict the needs of the 








team and the task, be able to adapt to changing 
environments, and coordinate with one another (Cannon-Bowers 
& Salas, 1991). Additionally, a shared mental model allows 


team members to draw upon team —- based knowledge to 





determine the best courses of action that are consistent and 
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coordinated with the actions of their teammates. 
Furthermore, sharing mental models allows team members to 


predict the behavior and resources required by the other 








team members more accurately. In human-automated teams the 


building of an accurate shared mental model may present new 








challenges to team effectiveness. 


B. COMMUNICATION 


Teams do not perform in a vacuum; communication is 


necessary to share information, coordinate efforts and 





complete tasks. To function effectively, a team must act as 





a coordinated information processing unit, maintain 
Situational awareness, and make collaborative decisions, all 
of which require some form of communication. Because 
communication requires time, effort and cognitive resources, 


there is an ‘overhead’ associated with its use. 





Christoffersen & Woods (2002) suggest that knowledge derived 


from communications in human systems is often derived at 





relatively low cost. In human-to-human cooperative work, 








people continually work to build and maintain a common 


understanding in order to support the coordinated problem 





solving efforts. Mutual knowledge of each others’ actions 





and abilities supports efficient and effectiv £FOLrtS. 





Another manner in which human to human interaction can 


be relatively low cost is in open work environments where 








individual team members can just observe the actions of 


others. The open nature of the environment allows team 





members to mak intelligent judgments about what actions 





have been taken, what actions are necessary and when they 


should be taken without any explicit communication. However, 





when it comes to automated team members, this information no 
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longer comes at low cost because automation often lacks 


intentionality. Human team members are often unaware of what 








the automation is doing, intends to do, and whether or not 








the tasks are conducted in a correct and efficient manner. 





In order to lower these communication costs, the automation 


must specifically be designed to generat eel shared 








understanding needed to support cooperative work. 





Sycara and Lewis (2004, p. 204) suggest that “the 


greatest impediment to assisting human users lies in 





communicating user intent to an agent and making the agent’s 








results intelligible to the human”. In today’s cases, the 








limiting factor in human-agent interaction is the users’ 





ability or willingness to communicate, organize, and 
interpret the machine’s response to satisfy them. Monitoring 


and evaluating is also more difficult because of increased 





flexibility and autonomy of the automation. The degree of 


this difficulty changes with the agent’s role. According to 





Sycara and Lewis (2004), there are thr possible roles for 





automated agents in human teams. The first is to support the 


team members in completion of their own tasks. The second is 





to assume the role of an equivalent team member by 


performing the reasoning and tasks of that human teammate. 





In this role, issues associated with communication and 
coordination are very relevant because not only does the 
automation have to perform its own tasks, but also must 


communicate to share information about its actions, goals, 








and progress. The third possible role is to support the team 
as a whole by facilitating communication, allocation of 


tasks, and coordination among team members. 
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The measurement and analysis of communication frequency 
and type has been an ongoing area of team research because 


of the critical role that communication plays in team 














ffectiveness. Som researchers hav developed and 





validated rating scales for assessing such communications. 


Macmillan, Entin, and Serfaty (2004) have developed and used 








several measures that explicitly link the team’s knowledge 
about each other and the shared tasks with the frequency and 


type of their communication. Their measures are based upon 








the theory that a shared mental model and shared awareness 


of information amongst team members results in the ability 











to coordinate implicitly, resulting in more efficient (lower 
overhead) communication. One such method proposed is the 


anticipation ratio, a measure of communication efficiency 





that has proven to b ffective for several different types 








of teams (MacMillian, Entin, & Serfaty, 2004). Their method 





calculates the ratio of the number of communications 








transferring information to the number of communications 


requesting information. Values greater than one indicate 





that communications were ‘pushed’ (information was. sent) 
among team members more frequently than requests for 


information were received indicating that the team was able 








to anticipate the need for information of the other team 





members. Values less than one indicate that information was 


‘pulled’ (requested) more frequently than it was’ sent. 








Miller and Shattuck (2003) have suggested another set of 





criteria for measuring communication between team members. 
They proposed 13 categories of communication: perception, 
comprehension, projection, pull, response to pull, push, 
decision/tasking, decision request, coordination, 
coordination request, coordination transfer, and 
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acknowledgement. These different categories can also be used 


to compare and contrast team communication frequency and 





type. Comparing communication types and frequency in human- 
automated teams increases the complexity of the analysis 
because the automation often uses a unigue language and its 


true intentions are often unknown. 


Cc. TRUST 


“Trust refers to the expectation of, or confidence in, 
another and is based on the probability that one party 
attaches to co-operative or favorable behavior by other 


parties” (Madhavan & Wiegmann, 2007, p. 290). Review of the 











trust literature suggests that trust is a multidimensional 
factor that generally follows patterns established by 
sociologists. Barber (1983) explains trust as a compromise 
between persistence of natural and moral laws, technical 


competence, and fiduciary responsibility. Regarding the 








integration of humans with automation, research has shown 





that these concepts could be associated with the 
reliability, predictability, and competency of the 
automation (Dassonville, 1996). Another sociologist, Rempel 




















(1985), explains trust in three dimensions: predictability, 








dependability, and faith. Predictability is the most 
concrete component of trust and depends on the stability of 


performance over time (Madhavan & Wiegmann, 2007). 





Dependability is based on the characteristics of the human 


in the system and is reflected in the level of confidence 





one has with the automation. Faith is based on beliefs 
about the future behavior or perceived accuracy of the 


automation and may be reflected in the person’s willingness 





to continue to use the automation. 


15 


Sheridan (2002) modifies the concept of trust in 


regards to human-automation interaction. He distinguishes 





between trust as an ‘effect’ of operators or outcome of 


certain automated characteristics and trust as a ‘cause’ of 





the human’s behavior when utilizing automation (Madhavan & 


Wiegmann, 2007). This concept may dictate whether the 





automation is used and the method of employment. Trust in 
the context of automation was identified by Lee and See 
(2004) on three general bases: performance, process, and 
purpose. Performance describes ‘what’ the automation does 


and refers to the operation of the automation. More 





specifically it refers to the competency as demonstrated by 








its ability to achieve th operator’s goals. It includes 








such characteristics such as reliability and predictability. 


Process describes ‘how’ the automation operates; it is 


the degree to which the automation actions are appropriate 





for the situation. Process as a basis of trust tends to 








reflect qualities and characteristics attributed to the 











automation that will lead to a concept similar to Sheridan’s 





(1992) statement that operators will tend to trust 





automation that can be understood and is likely to achieve 


the operator’s goals (Lee and See, 2004). 


Purpose refers to the degree to which the automation is 





used as compared to the intent of the designer. It 
describes ‘why’ the automation was developed and corresponds 
to the faith and benevolence of the operator. Lee and See’s 


first two bases of trust (i.e., process and performance) 





tend to correspond to Rempel’s (1985) assertions of 


predictability and dependability and their third dimension 
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(purpose) roughly corresponds to the component of faith and 





benevolenc described in the Rempel model (Madhavan 6& 


Wiegmann, 2007). 


Although past research has generally supported the 


concept that machine reliability predicts trust in 











automation, Merritt and Ilgen (2008) contend that the user’s 


personality and perceptions of automation also play a vital 





role in determining automation utilization decisions. They 





imply that an individual with a greater disposition towards 





trusting other people tends to display greater levels of 

















trust when interacting with automation. Studies such as 
Reeves and Nash (2006) show that humans respond socially to 
technology, and reactions to computers and automation tend 


to be similar to human collaborators (Lee & See, 2004). 











Merritt and Ilgen (2008) use this concept to suggest that 


Since extroverts tend to be more sociable than introverts, 





they tend to trust automation more frequently; so biases in 








social behavior may have an effect on reliance of 
automation. Their study involved an X-Ray screening task 
and showed significant results to support their assertion. 





Another operator personality characteristic - self- 
confidence - may also be related to automation use as 
described by Lee and See (2004). Self-confidence is a 


critical factor in decision making and has effects on trust. 


They suggest that as operator confidence increases, use of 





automation decreases. The opposite is also true: low self- 


confidence directly relates to increased reliance on 











automated functions. 
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Even though trust between human team members and trust 
between human and automated team members has many 


Similarities, some studies suggest that there are important 








differences between them. Trust between people is generally 





part of a social exchange relationship resulting from 


repeated interactions between people. There is a 








symmetrical relationship to this interpersonal trust as each 
party is aware of the other’s behaviors, intentions, and 


trust (Lee and See, 2004). But since automation often does 





not explicitly state its intentions, there is no symmetry in 











the relationship; therefore, people tend to trust automation 








through a different process. 


Another research study (Lewandowsky, S., Mundy, M., & 
Tan, G., 2000) found that delegation to automation was 


different than delegation to human counterparts. They found 





that delegation to other humans, but not automation, was 


based on people’s assessment of how others perceive them. 











If people perceive their own trustworthiness to be low among 


other people, they are more likely to delegate tasks. 





Additionally they found that in delegation to automation, 
rather than humans, the degree of trust played a more 
critical role. One possible explanation is that in human to 


human partnerships, the ultimate responsibility is perceived 








as being shared as compared to the perception of ultimate 
responsibility resting with the operator in a human- 
automation partnership (Lewandowsky, S., Mundy, M., & Tan, 


G., 2000). 


The final differenc between interpersonal trust and 





trust in automation deals with the attribution process. 





Interpersonal trust tends to evolve in accordance with some 
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of the previously discussed models. Initially, trust is 











based on performance (reliability). Eventually, trust is 
based on dependability or process. Ultimately trust is 
based on purpose or faith. Trust in automation generally 


follows the opposite progression in which faith is paramount 





early, followed by dependability, and ending with 





predictability (Lee & See, 2004). Substantial evidence 
exists to support the conclusion that trust in automation is 
meaningful and useful in understanding reliance on 


automation, but the lack of a symmetrical relationship, lack 





of intentionality, and differences in progression of trust 











indicate that caution must b xercised when attempting to 











extrapolate results from human-human trust to the human- 











automation trust relationship. 


D. TEAM EFFICACY 








In 1986, Bandura defined self-efficacy as “people’s 





judgments of their capabilities to organize and execute 


courses of action required in attaining designated types of 





performances” (Karrasch, 2003). Later Bandura (1997) 
suggested that self-efficacy reflects an individual’s belief 


is his own capabilities to pursue a course of action to meet 








given situational demands. There are three important 


characteristics of self-efficacy according to Gist & 








Mitchell (1992). Self-efficacy: 
° involves a judgment or comprehensive review of the 
perceived capabilities of the individual 


performing the task 


e has a motivational component 
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e is dynamic and changes over time in response to 


new experiences and information. 





Self-efficacy also has a powerful effect on task 








performance (Durham, Knight, & Locke, 1997). There is clear 
evidence that efficacy has an effect on performance and is 


not simply a matter of past performance being correlated 





with future performance. Since people with higher self- 
efficacy tend to believe they can accomplish more, they tend 


to set higher goals. Accomplishment of higher goals tends 





to build self-confidence, which leads to greater efficacy. 


Finally Woods and Bandura (1989) suggest that higher self- 





efficacy leads to the use of mor ffectiv strategies, 


which tend to be more successful, thereby increasing 








ffectiveness, which in turn increases self-confidence, that 








results in an even greater level of efficacy. 





The role of efficacy on performance may shift slightly 





when transitioning from individual performance to group 
performance. Therefore, collective efficacy at the group 


level has been suggested as a construct parallel to self 





efficacy at the individual level (Durham, Knight, & Locke, 








1997). Bandura has argued that perceived collective efficacy 
is an emergent group-level property, not just the sum of the 
efficacy beliefs of the individual members (Karrasch, 2003). 
Collective efficacy may reflect the shared beliefs of the 
group’s members in their ability to accomplish the task at 
hand. Bandura (1997) argued that collective efficacy 
influences the level of persistence, the effort, and the 


actions taken by group members in an effort to accomplish 








group tasks. Durham, Knight and Locke (1997) state that 











efficacy can be expanded to teams to the extent that 
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individual team members agree that their team can perform 





successfully at any given task. This leads to group 
efficacy which Gibson (1999) defines as the extent to which 
a group believes it can accomplish a task through 
concentrated and coordinated effort. Since efficacy tends 
to affect team dynamics as a team develops and gains 


confidence, it tends to affect group performance in a 








positive fashion. Indeed, researchers have established that 


group efficacy is a meaningful and measurable group 











attribute and is positively correlated to group 
ffectiveness (Tyran & Gibson, 2008). Bandura (1997) 
suggested that group efficacy is often related to the level 





of effort a group expends because it relates to what the 
group thinks it can accomplish. Therefore group efficacy 


tends to also affect group motivation and confidence. This 





type of positive group efficacy may be more difficult to 





establish in human-automated teams because the human players 
are often not as familiar with the competency of the 


automation. 


E. SUMMARY 


Many factors must be considered in the formation of 


successful teams. Team communication type and frequency, 





trust between team members, team shared cognition and team 





efficacy are just a few of the factors that may affect team 


performance. 





Research suggests that team performance among mixed 


human and automated teams may not be driven in the same 





methodology as teams consisting of only human members. In 
fact, (Fiore et al., 2005) argue that it is not clear what 


information human-agent teams use in order to effectively 
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perform a given task, and how the presence of non-human team 
members alters what has traditionally been considered to be 


the requirements of effective teams. 








Based on previous research, this thesis hypothesizes 
that teams consisting of only human members will communicate 


in a different fashion than human-automated teams. The 








literature suggests that the nature of communication between 














automation and humans is not the same as how humans 
communicate with each other. The second hypothesis 


asserted is that team efficacy will be higher for the teams 





without the automated agent. Team efficacy involves trust 
and the ability to understand the nature of each team 
member’s actions and intentions. Most automation lacks the 
ability to fully express intentionality, which often results 
in a loss of confidence among the human team members. The 
third hypothesis is that the level of trust will be higher 


in the teams consisting of only human members. The 








literature suggests that LrUust between a human and 








automation, especially new automation, is not the same as 





trust between humans. 
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III. EXPERIMENTAL METHOD 


Measuring the nature of interactions between team 
members and overall team performance in combined human and 
automated teams can be challenging. Trust also becomes a 
major factor in performance, especially with partially 


automated teams. Some experiments have focused on the types 





of information shared; some have concentrated on the level 
of trust amongst team members, while others have been more 
concerned about the actual achievement of desired 
objectives. For the purpose of this study, team performance 
was examined as a function Of mission completion, 
communication and team efficacy. A computer-based 
microworld, C3Fire, was utilized as a tool to measure team 


performance in a controlled environment. 


C3Fire was developed in 1993 as a research project ina 
collaborative effort between Rego Granlund and Henrik Artman 
at LinkdOping University in Sweden. C3Fire supports training 


and team research in a controlled environment (Overview. 





Retrieved October 22, 2008, from: 








http://www.c3fire.org/c3fire/overview/overview.en.shtml). 





In the C3Fire microworld, a firefighting scenario is 
generated on a 40 x 40 matrix of cells. Participants control 
three types of trucks (i.e., firefighting, water, and fuel) 


in a collaborative effort to extinguish the fire. The 





firefighting trucks extinguish the fire, the water trucks 





provide water to the fire trucks, and the fuel trucks 
provide fuel to all trucks. Participants need to manage 


their own water and fuel states throughout the scenario. 
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Other parts of the interface for the participants include a 





chat system, a unit information panel, a unit property 


panel, clock, and a map legend (see Figure 2). 





[ re_][_ coor mou Eater 









































Figure 2. C3Fire User Interface. (From: www.c3fire.org) 


A. PARTICIPANTS 


For this study, all of the participants were U.S. 
military officers at the Naval Postgraduate School. The 


confederate member was a research associate employed at the 





Naval Postgraduate School. 


Thirty-six participants (average age = 31.86, SD= 4.08) 
were assigned to twelve teams. Each team consisted of four 
members, three participants and a confederate. The 
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confederate was designated as the gas truck operator in 


every group. The participants consisted of 28 male members 


and 8 femal 





members ranging in age from 24 —- 39 years old. 


The participants included five Lieutenant Junior Grades, 





eighteen Lieutenants, five Captains, four Lieutenant 


Commanders, and four Majors. Four participants were in the 





U.S. Army, four were in the U.S. Air Force, 27 were in the 


U.S. Navy, and one was in the U.S. Marine Corp. Six teams 


were all male and the other six teams were of mixed gender. 




























































































Table shows team composition. 
Composition Composition 
Automation 
Human First First 
Teams Age, Sex, Rank Teams Age, Sex, Rank 
(1) 29, M, O-3 (2) 32, M, O-3 
37, M, O-4 35, M, O-4 
31, F, O-3 32, M, O-3 
(3) 25, F, O-2 (4) 35, M, O-4 
33, M, O-3 37,M, O-4 
30, M, O-2 35, M, O-3 
(5) 26, M, O-3 (6) 35, M, O-3 
26, F, O-2 36, F, O-3 
24, M, O-2 31, F, O-3 
(7) 29, M, O-3 (8) 31, M, O-3 
33, M, O-3 28, M, O-3 
39, M, O-4 36, M, O-3 
(9) 37, M, O-3 (10) 26, M, O-2 
28, M, O-3 28, M, O-3 
29, M, O-3 34, F, O-3 
(11) 38, M, O-4 (12) 30, M, O-3 
37, M, O-4 31, F, O-3 
31, M, O-3 32, F, O-4 
Table 1. Team Composition 
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In order to control for prior knowledge of other team 
members, it would be desirable to form groups whose members 


had never met, but some individual team members were known 








to each other. The Naval Postgraduate School student body 


consists of approximately 1,900 students making it virtually 





impossible to form groups of students who had not served 





with each other previously or taken classes together. 





The experimental procedures were screened and approved 





by the Naval Postgraduate School Institutional Review Board 











— 


IRB), thereby meeting both the Department of the Navy and 





the American Psychological Association (APA) standards. All 








participants signed an informed consent form, which notified 


them of their rights as a participant in the experiment, and 








an exit debrief, form which informed them that intentional 





deception was necessary to complete th xperiment. 


B. PARTICIPANTS 


1. C3Fire 


The C3Fire program requires approximately 250 MB of 


free disk space and was run on a desktop personal computer 





server. All computers used by the participants and the 


confederate were identical. 











a. Server Specifications 
e 24” Dell monitor 
e Dell Optiplex 745 desktop computer 





Windows XP O/S 
Intel Core 2 CPU, 2.66 GHz, 3.0 GB RAM 








Broadcom NetXtreme 57xx Gigabit Network 
Controller 


® Netgear DS108 10/100 Mbs Dual Speed Hub 
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b. Participants’ Computer 











° 20” Dell monitor 
e Dell Optiplex 745 desktop computer 
7 Windows XP O/S 
7 Intel Core 2 CPU, 2.66 GHz, 3.0 GB 
RAM 
7 Broadcom NetXtreme 57xx Gigabit 


Network Controller 

















7 Netgear DS108 10/100 Mbs Dual Speed 
Hub 
VARIABLES 
1. Independent Variables 
e Group Composition - A team consisted of four 


human members or a team with three human 
members and one ‘automated’ agent. 


2. Dependent Variables 


e The number of messages sent per minute 
between team members during the scenario. 








° The type of messages sent between team 
members during the scenario. 





° The responses to the Team Efficacy 
Questionnaire. 
e C3fire task performance (the number of cells 


in the 40 x 40 grid that are burned, have 
been extinguished, or remain on fire at the 
conclusion of the scenario). 


e The number of commands issued to a truck by a 
team member who was assigned a different 
role. 
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D. PROCEDURE 


Participants were solicited through various methods and 


assembled into teams of three based on their availability. 








The experiment was conducted in an enclosed lab space in the 








Human Systems Integrations Laboratory (HSIL) at the Naval 
Postgraduate School. The lab space contained the C3Fire 
server and four client computers. The fifth client 


computer, used by the confederate, was located in a separate 











private lab area and was unknown to the participants. 


The participants and the confederate arrived at the 








HSIL independently and were allowed to socialize until the 


experiment started. All members were given consent forms to 





sign followed by PowerPoint training on the C3Fire program 








and the parameters of th xperiment. Th xperimenter 
explained that two scenarios would be run, one with four 
human players and the other with three human players and an 
intelligent automated agent playing the role of the gas 


truck operator. The methodology for communicating with the 








automated agent was also explained. Next, all four 


participants picked a computer station at random and a 





training scenario was conducted using the C3Fire program. 





Information to provide clarity among team member roles and 





communication instructions for the automated agent was 
provided at each client station. Figure 3 shows the C3Fire 


client computer setup. 
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Figure 3. C3Fire Client Computer Station. 


Since actual roles had not been assigned at this point, 





all members were allowed to practice all four possible roles 
on the team. At the completion of the training, each 


participant was randomly assigned a permanent role on the 





team. The confederate was always assigned the role of the 
gas truck operator. The participants were told the role of 


each of the other team members. 





At this point, the group was notified which team 
composition would be utilized for their first scenario: 
either four human team members or three members and the 
automated agent. Next all team members completed an 


electronic Team Efficacy Questionnaire (TEQ). The TEQ was 
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generated after consultation with a Professor of Management 
from the University of Nebraska-Lincoln, Dr. Bruce Avolio. 
He suggested appropriate questions for the TEQ, which were 
then modified to align with the C3Fire scenario (see 


Appendix A). 


For the condition with four human members, the first 
scenario was then started. The scenario lasted until either 
the fire was extinguished or twenty minutes had elapsed. 


Another TEQ (see Appendix  B) was completed and the 





confederate was dismissed to go back to work. In reality, 





the confederate left the enclosed lab area and went to the 





fifth C3Fire client computer located in a separate lab 
space. The second scenario was then conducted utilizing the 


confederate as the automated agent followed by another TEQ. 





Th xperiment was counter balanced such that half the 
groups started with four human team members in scenario one 


and the other half started with the automated agent. Both 





scenarios were similar and the behavior of the confederate 


was scripted to be the same for both scenarios. 





Following the two scenarios all team members met face- 


to-face for a recorded exit debriefing. At the conclusion of 








the verbal debriefing, the true role of the confederate was 





revealed and the participants signed forms indicating they 


had been informed of the intentional deception. 


30 


IV. RESULTS 


A. TEAM COMMUNICATIONS 


dy. Communication Density 


Communication density was analyzed by examining the 


number of messages sent by each team during each scenario. 





C3Fire provides a chat box tool, Similar to instant 


messaging, that was the sole form of communication between 





the participants. During scenarios in which the confederate 





was playing as the intelligent agent gas truck operator, he 








did not generate any messages. During scenarios in which he 





was acting as the human gas truck operator, he sent a few 


messages to maintain his credibility. None of the 











confederate’s messages were included in the analysis of the 





communication density data. The reason they were removed is 





because this analysis was conducted to determine whether th 











participants interacted differently with one another in the 
two conditions (1.e., confederate acting as human or 


intelligent agent). Figure 4 shows the average number of 








messages sent between team members per minute for each team 
type. The human team type sent an average of 2.51 (SD=.78) 


messages per minute while the automated teams sent an 





average number of 2.49 (SD=.92) messages per minute. A t- 








test was conducted to determine if the average number of 








messages per minute was the same across both team types. 


The difference in the number of messages sent by team type 





was not statistically significant (t(11)=.09, p=.93). 
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Figure 4. Communication Density for each Team Type. 


Communication density was then analyzed based on the 
number of messages sent to the human confederate per minute 


versus the number of messages sent to the automated 





confederate per minute. Figure 5 shows the average number 
of messages sent to each type of confederate per minute. The 


human confederate received an average of 0.88(SD=0.39) 








messages per minute while the automated confederate received 





an average of 0.97(SD=0.54) messages per minute. A t-test 
was conducted to determine if the average number of messages 
sent per minute to the human confederate was the same as the 


average number of messages sent to the automated 





confederate. Again, no statistical significance was found 





for the average number of messages sent per minute based on 


the type of confederate (t(11)=1.22, p=.25). 
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Figure 5. Communication Density to the Confederate. 


2. Communication Type 





The messages between team members were then categorized 
into six of the thirteen types identified by Miller and 
Shattuck (2003): projection, push, decision/tasking, 
coordination, acknowledgment, and pull. A seventh category, 
chatter, was added to account for communications that were 


not relevant to the scenario. Table 2 shows the average 





number of each type of message sent per minute by team type 
and to each type of confederate. Further analysis was 
conducted by applying t-tests to compare the means for each 


type of message for all team members. Another set of t- 





tests was conducted on just those messages sent to the 





confederate. The t-tests revealed one significant result. 
ao 


The number of messages pushing information was different 





between the two types of confederates (t(11)=2.55, p=.03). 





The human confederat received an average of .18(SD=.03) 








messages per minute, while the automated confederate 





received and average of .07(SD=.01) messages per minute. 





Table 3 shows the p values for the t-tests for each 


communication type between all team members and to the 





confederate. 
















































































Total 
Total Human Automated To Human To Automated 
Type of Message Team Team Confederate Confederate 
Projection 0.09 0.11 0 0 
Push 0.62 0.78 0.18 0.07 
Decision/Tasking 1.1 0.8 0.38 0.67 
Coordination 0.56 0.56 0.32 0.21 
Acknowledgement 0.01 0.08 0 0 
Chatter 0.09 0.12 0 0 
Pull 0.04 0.04 0 0.01 
Table 2. Average Number of Communication by Type per 
Minute. 
All Team Confederate 
Type of Message Members Only 
Projection 0.75 Insufficient Data 
Push 0.19 0.03 
Decision/Tasking 0.07 0.1 
Coordination 0.99 0.21 
Acknowledgement 0.19 0.34 
Chatter 0.07 Insufficient Data 
Pull 0.37 0.34 
Fable 3 bp Values for t-tests: by Communication Type. 
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3°. Participants’ Comments 





During the face to face debrief, and before the 


identity of the confederate was revealed, participants were 





asked to discuss their xperienc participating in this 





study. Th researcher did not direct the debrief, or 





specifically address any topic; the participants were 





allowed to discuss any aspect of the study they desired. The 
confederate was also present at the debrief, but did not 


reveal his identity or provide any comments that would 





influence the discussion of the participants. The exit 





debriefs were recorded and later analyzed for any comments 





specifically related to the automated agent. The comments 





related to the automated agent were overwhelmingly negative; 


twenty-one comments (84%, C1:69.7-98.3%) indicated a 








negative perception of the automated agent, while only four 





(16%, CI:1.6-30.4%) indicated a positiv xperience with the 














automation. A typical comment was ‘I did not trust the 











automated gas trucks.’ See Appendix C for all comments 


referencing the automated agent. 


B. TEAM EFFICACY QUESTIONNAIRE 


Question number five from the TEQ, (I had confidence in 
the gas truck operator’s ability to properly perform the 
role), was analyzed from three different aspects. First, an 
order effect (human or automation first) was evaluated. 
Next, a team type effect (human or automated confederate) 
was investigated. Finally, an effect based on transitioning 


from one type of team to the other (human to automation 





versus automation to human) was evaluated. In addition to 
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those three, an analysis of the transition effect from pre 


TEQ to post TEQ was also conducted for question number five. 


1. Order Effect 





In this study, the order of presentation of the 





scenarios and the role played by the confederat wer 

counterbalanced. Using question 5 from the TEQ, (I had 
confidence in the gas truck operator’s ability to properly 
perform the role), a two-tailed Sign test was conducted to 


determine if the participants’ confidence levels of the gas 





truck operator’s ability was influenced by the order of the 


scenarios. Figure 6 shows the average response to question 5 








based on the order of the scenarios. The teams that 


completed the human scenario first and the automated 





scenario second had an average score of 3.69(SD=1.01). The 
teams that completed the automated scenario first and the 
human scenario second had an average score of 3.67(SD=1.31). 
A two-tailed Sign test was conducted and no significant 
difference was found for confidence in the gas’ truck 


operator’s ability (z=0.00, p=1.00). 
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Figure 6. Average Response by Order of Scenario. 
2. Team Type 





In this study there were two different types of teams. 
The first type of team consisted of three participants and 
the confederate, who played the role of another human 
participant. The confederate was assigned as the gas truck 
operator in every team. The second type of team consisted 
of the same three participants and an ‘automated agent.’ 
The ‘automated agent’ was actually the confederate who was 


assigned the role of the gas truck operator and played the 





C3Fire game from a remote location. A two-tailed Sign test 


was conducted on responses to question 5 of the TEQ, (I had 
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confidence in the gas truck operator’s ability to properly 
perform the role) to determine if there was an effect of 


perceived team composition on confidence level in the gas 








truck operator’s ability. Figure 7 shows the average 
response to question 5 for each type of team. The human 
teams had an average score of 4.17 (SD=0.74), while the 
perceived automated teams had an average score of 3.19 


(SD=1.31). A significant difference was found in the 





confidence level of the gas truck operator’s ability based 


on team composition (z=3.14, p=0.0025). 





TEQ Response: ! had confidence in the gas truck operators ability 











Average Response 








All Human Groups All Automated Groups 
Team Type 











Figure 7. Average Response by Group Type. 
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3. Transition Effects 





In this study the participants were told either their 





first scenario would be conducted with a human participant 
or with the automated agent. Prior to playing with this 


team composition, the participants were asked to take a pre- 





scenario TEQ (Attachment A). Immediately after the scenario 


ended the participants were asked to take the post scenario 





TEQ (Attachment B). The teams that played in the human 
condition first had an average score for question 5 of 3.94 
for the pre-scenario TEQ and an average score of 4.06 for 


the post scenario TEQ, resulting in an average increase in 





confidence in the gas truck operator’s ability of .12. A 
two-tailed Sign test indicated no significant difference 


(z=.32,p=.625) The teams that played in the automated 





condition first had an average pre-scenario TEQ score for 


question 5 of 3.72 and an average score of 3.06 for the post 








scenario TEQ, resulting in an average decrease in confidence 


of the gas truck operator’s ability of .66. A two-tailed 
































Sign test revealed a significant difference (z=2.66, 
p=.0039). Table 4 summarizes the results. 
Pretest Human Average Post Test Average Score for Human 
Score Scenario First Nethange 
3.94 4.06 0.12 
Pretest Automation Average Post Test Average Score for 
Score Automation Scenario First Nevehange 
3.72 3.06 -0.66 
Table 4. Transition Effect from Pre scenario to Post 
scenario. 


After each team type finished their first scenario, the 





other team type was utilized to complet Eh second 





scenario. Immediately upon completion, another post 
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scenario TEQ was completed. The teams that transitioned 
from a human player to the automated agent scored an average 


of 3.33 for question 5 on the post scenario TEQ, resulting 





in an averag decreas in confidence in the gas truck 








operator’s ability of .73. A two-tailed Sign test revealed a 
Significant difference (z=2.88,  p=.002) The teams that 
transitioned from an automated agent to a human player 
scored an average of 4.28 for question 5 on the post 
scenario, resulting in an average increase in confidence in 


the gas truck operator’s ability of 1.22. A two-tailed Sign 





test revealed a significant difference (2z=3.89, p=.0001) 





Table 5 summarizes the results. 





























Post Test Average Score for Post test Average Score for NarChanae 
Human Scenario First Automation Scenario Second g 
4.06 3.33 -0.73 
Post Test Average Score for Post test Average Score for Human NahGhanae 
Automation Scenario First Scenario Second g 
3.06 4.28 1.22 
Table 5. Transition Effect of Changing Team Type. 
C. TEAM PERFORMANCE 
dis C3Fire Cell Status 


At the conclusion of each scenario, the cells on the 





C3Fire matrix that caught on fire at any time during the 
scenario could be in one of three states: still on fire, 
extinguished, or burned out. Still on fire indicated the 


team never attempted to fight the fire in that cell or time 





expired prior to that cell being extinguished, extinguished 











meant that the team successfully used the trucks to put the 





fire out; and burned out meant that the team did not 
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successfully extinguish the fire, but the cell had been on 
fire for a long enough period of time to stop burning. 


Human teams had an average of 18.00 (SD=27.52) cells still 





on fire, an average of 27.33 (SD=8.56) cells extinguished, 
and an average of 106.33 (SD=102.80) cells that had burned 
out at the conclusion of the scenario. Automated teams had 
an average of 24.67 (SD=27.82) cells still on fire, an 
average of 33.67 (SD=12.56) cells extinguished, and an 
average of 148.08 (SD=120.42) cells that had burned out at 





the conclusion of the scenario. Figure 8 shows the average 





final cell status for each type of team. A two-tailed Sign 
test was conducted to determine if team type had an effect 


on the cell status at the end of the scenario. No 





statistical significance was found for number of cells on 
fire (p=.45), number of cells extinguished (p=1.0), or 


number of cells burned out (p=.15). 
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Figure 8. Average C3Fire Cell Status at Scenario Completion. 
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2. Team Success Rate in Completely Extinguishing the 
Fire 
The C3Fire scenario used for this study produced a fire 
that could be extinguished by an effective and coordinated 


team. Fourteen of the twenty-four (58.3%) sessions ended 





prior to the twenty minute time limit because the fire had 
been completely extinguished. Figure 9 shows the number of 
teams able to successfully extinguish the fire by team type 


and order. 
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Figure 9. Team Success in Extinguishing Fire. 


3:3 Scenario Length 


The C3Fire session ended ither at the artificial 





twenty minute time limit or when the team successfully 





extinguished the entire fire. Ten of the twenty-four 
(41.7%) sessions lasted the entire twenty minutes. Table 6 


shows the length of each session by team type and order. 
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Team Type and Order Team 1 Team 2 Team 3 Team 4 Team 5 Team 6 
Human First 20 20 7.95 7.72 8.1 11.83 
Human Second 8.55 7.2 8.15 20 20 8.52 
Automation First 20 10.52 20 20 20 20 
Automation Second 8.17 8.52 8.3 20 14.35 8.12 
Table 6. Session Lengths in Minutes. 





The average scenario length for human teams was 12.34 





minutes (SD=5.77), while the average scenario length for 


automated teams was 14.83 minutes (SD=5.65). Figure 10 





shows the average scenario length by type of team. A t-test 





was conducted to determine if the average scenario length 
was different for each team _ type; no statistically 


Significant results were found (t(11)=1.07, p=.30). 
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Figure 10. Average Scenario Length by Team Type. 
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D. TASKING NON-ASSIGNED UNITS 


During the C3Fire training process, participants were 





instructed to move only the units assigned to their role, 





unless there was a compelling reason to task other 


participant’s units. C3Fire is able to determine which 








participant tasked which units, thereby enabling researchers 





to determine the number of times a unit was tasked by a 








participant other than the one with the primary 


responsibility for that resource. Table 7 shows the total 





number of taskings to other player’s assets by player role 


and team type. A paired t-test was conducted on the total 





number tasks issued to units by a player other than the 
designated player. The results were statistically 
Significant (t(11)=2.58, p=.03) indicating that the number 


of tasks issued to non-assigned units was different between 





the two types of teams. 






































Human Automated 
Player Role Teams Teams 
Command & 
Control 3 90 
Fire Truck 19 62 
Water Truck 33 34 
Gas Truck 1 2 
Total 56 188 
Table 7. Number of Tasks Issued by Participants to 


Resources of Other Participants. 


Further analysis was conducted for tasking of just the 
gas trucks by other participants. Table 8 shows the number 
of tasks issued by other participants to the Gas Trucks. A 
paired t-test was conducted on the total number tasks issued 
to the gas trucks by a player other than the gas truck 


operator. The results were statistically significant 
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(t (11) 


=2.41, p=.03), 


indicating that the number 


of tasks 


issued to the gas trucks by other participants was different 


between the two types of teams. 


Table 8. 


Figure 11 shows a combination chart of tasking of 


gas truck operator by team type and the total number of 





type. 


role 


condition, as well as how much each role tasked th 


taskings 


























Human Gas Automated 
Player Truck Gas Truck 
C+C 0 64 
Fire 5 34 
Water 7 5 
Team 2 103 








Number of Tasks 





Issued by Other Participants 


the Gas Trucks. 





The chart indicates the 





trucks. 
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by other than non-assigned participants by 


to 


the 


team 





team 
total number of times each 
tasked the gas tuck operator in the human and automated 


other 
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Figure 11. Non-Assigned Member’s Number of Taskings to the 


Gas Truck and the Total Team by Team Type. 
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V. DISCUSSION 


A. HYPOTHESIS ONE 


The hypothesis that teams consisting of only human 


members will communicate in a different fashion than teams 





comprised of human members and an automated agent is 


partially supported by this research. The difference in the 





number of messages sent by team composition was not 





Statistically significant. Furthermore, the number of 
messages sent to the confederate for each team type was not 


significantly different. One possible explanation why there 





were no differences found in the number of messages sent is 





the low number of messages actually sent. Since all players 
were able to view the entire map and the actions of the 


other players, the use of chat to communicate intent was 





often unnecessary. For example, when a player moved a truck 
to a new location on the map, an indication of the new 
intended location for that truck was displayed, thereby 


eliminating the need to inform the rest of the team. 


Additionally, the messaging tool diverted participants’ 


attention and required players to spend their time sending 





and reading messages rather than managing their assets. 


Given this configuration, messages wer often overlooked 





when participants experienced a heavy workload. Finally, 


Since all scenarios were not the same in duration (because 








some teams extinguished the fire more quickly than others), 


the number of messages sent were analyzed per unit of time. 





The resulting analysis did not yield a significant 


difference. 
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The next step in the analysis was to identify the types 


of communications utilized by each team type and how each 








type of team communicated to the confederate. No differences 


were found in the types of communications between each team 





type (1.e., projection, push, decision/tasking, 


coordination, acknowledgment, pull and chatter). Only one 





statistically significant difference was found in the 





analysis of the type of communications sent to the 


confederate. The number of messages ‘pushed’ to the human 








confederate was significantly greater than the number of 





messages ‘pushed’ to the automated confederate. A message 
labeled as ‘push’ is sent to a player without it being 


specifically requested. This information could then be 





utilized to build awareness or make decisions. The teams 
playing with the human confederate may have thought it 
appropriate to pass this type of information to a human 
player, while the teams playing with the automated 


confederate may have assumed that the automated player 





already had access to the information. 


There were other examples of the automated player being 





treated differently than the human player. Participants 





attempting to coordinate with - or direct the actions of - a 


human player would allow a reasonable time lag for the human 





to process the information and take action, but when these 
same participants thought they were interacting with an 


automated agent, they desired an immediate response. When 





interacting with another human player it was also acceptable 





for the human to continue the current task and then address 








the new task, but when the participants interacted with the 








automation, they desired an immediate stop in the 











automation’s current actions to address the new task. One 
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possible suggestion is that the participants believed th 
automated agent could process in parallel, but the human 


could only process or perform one action at a time. 


This research effort only partially supported the 





hypothesis that human-human teams communicate differently 


than human-automation teams. Christoffersen and Woods 








(2002) suggest that the ‘overhead’ associated with 





communication in a human system incurs a relatively low 
workload cost in open work environments, but utilizing 


automation in the same situation will increase the cost. The 





C3Fire microworld utilized in this study did not take 


advantage of an open work environment; the participants were 





not allowed to communicate through any method other than the 





chat box. This study was not able to determine the benefit 





of open work environments to human communication, but does 
support the concept that closed work environments drive up 


the cost for communicating with automated agents as compared 





to human agents. 


The work of Sycara and Lewis (2004), which suggests 
communication is one of the biggest challenges in human- 
automation teams, is strongly supported by this study. 


Their belief that the users’ ability and willingness to 











communicate with the automated agent is a limiting factor 








held true in this study. While under heavy workload, the 








participants appeared to put minimal effort into the task of 


communicating with the automated agent. 
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B. HYPOTHESIS TWO 


The second hypothesis, that the human teams would have 
a higher team efficacy, was supported by this research. The 
Team Efficacy Questionnaire (See Appendix B) was analyzed 
for differences in confidence level for each team role. No 


Significant differences were found for the roles of command 








and control, fire truck operator, or water truck operator. 
However, question number five, (I had confidence in the gas 
truck operator’s ability to properly perform the role), 


provided significant results. When analyzed for an order 








ffect, ither human-human or human-automation team first, 





no significant results were found indicating that order had 
no effect on the perceived confidence level of the gas truck 
operator. When question number five was analyzed for the 


type of team, containing either a human or automated agent, 





Significant differences wer found with respect to the 





perceived confidence level of the gas truck operator. The 


participants rated the human confederate significantly 





higher than they rated the automated confederate. 


The style of game play of the participants appears to 


be reflected in the TEQ. The participants were much more 











likely to override the actions of the perceived automation 
than the human. When the gas truck was under human control, 


there were only 12 instances (the fewest of any truck) where 





the participants chose to issue a command to the gas truck. 
When the gas truck was controlled by the perceived automated 
agent, there were 103 commands (the most of any truck) 
issued to the gas truck by the participants. The difference 


in number of commands issued suggests that the participants 





had more confidence in the human gas truck operator. Some 
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participants also elected to ask the human gas truck 





operator for permission to move a gas truck, but no 





participant ever attempted to communicate with the automated 


agent prior to moving a gas truck. 


Bandura*‘s (1997) assertion that group efficacy may be 
more difficult in automated teams because the human players 
may not be familiar with the level of competence with the 


automation is supported by this research. Most of the 





participants were unsure of the level of competency of the 


automation and chose to monitor its actions very closely. 





Deviations from what the human player believed to be the 








best course of action often resulted in the participant 
interfering with the automation. This breakdown in roles and 
responsibilities affected group efficacy, and appears to be 


linked to the results of the TEOQ. 





Sinc team fficacy is often more difficult to 
establish and maintain in human-automation teams, those who 


design complex systems need to give due consideration to the 











method by which human and automated agents interact with one 





another. A possible area for improvement would be to provide 
the automated agent with an ability to communicate in a 


manner similar to human communication. The chances would be 





better that the automated agent would be understood and 


accepted as an equal team member. 


Cc. HYPOTHESIS THREE 


Based on the comments from the debriefs with the 





participants, such as ‘I did not trust the automated gas 


trucks’, it was guite apparent that the level of trust in 





the automated gas truck operator was considerably less than 
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that of the human gas truck operator (See Appendix C). When 








the confederate played the role of the automated gas truck 





operator, there was a statistically significant increase in 





the number of commands sent to all trucks by participants 
not responsible for those assets. A likely reason this 
occurred is that once trust in the automation broke down, 
the team lost its sense of roles and responsibilities. The 
players then began to spend energy managing other assets, 


which left their own assets partially unattended. The second 





possible contributing factor is that the participants were 


heavily tasked. As participants began to compensate for the 








automated gas trucks, they were unable to properly fill 





their assigned role causing other team members to intervene. 





Although not statistically significant, potentially due 





to the low number of messages, the teams playing with the 





automated confederat tended to send a greater number of 


decision/tasking messages to the confederate indicating the 





participants were making decisions and tasking the 











automation, not trying to provide information for the 











automation to make a decision. 


According to literature cited earlier (Christoffersen & 





Woods, 2003), feedback is a crucial element to the success 
of a human-automation team. Feedback is crucial since the 


human team members often lack an understanding of the 



































automation’s intentions. As the complexity of the 
automation increases, the level of feedback needs to 
increase. In this study a feedback mechanism for the 
automation was in place, but may have been too cumbersome to 


useful. This resulted in a breakdown in trust between the 





human team members and the automation. Since the appropriate 
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feedback mechanisms were not in place, most participants 





chose to strictly monitor the automation and intervene onc 


the barrier of trust had been violated. Trust needs to be a 











top consideration in the design of any automated system that 





will interact with human team members. Once the bond of 


trust is broken between th automation and the human team 








members, the team will los ffectiveness. 





This study expected a greater number of communications 


between the participants about the automated agent. 








Although most of the participants were concerned about the 








competency of the automated agent, they did not express that 








concern to the other participants. Additionally, the 





participants did not ask each other what their thoughts were 
about the automation. There were no collective efforts by 
the participants to evaluate the automated agent. One 


possible explanation is that the automated agent did perform 








at the same level as the human player, which did not warrant 


the need for any concern. Another possible explanation is 





that the participants were unsure of the automated agent’s 
abilities and chose to monitor it in an effort to gather 


more information to make a decision. 








When the automated agent’s performance was adequate, 


there was no real reason to ask for the rest of the 





participant’s thoughts. But when the performance of the 





automated agent became substandard, instead of communicating 





with the other human team members, the participants choose 


to just deal with the automated agent at a personal level. 





At this point the roles and responsibilities of the team may 


have broken down to the point where the participants felt it 








necessary to concentrate on only their actions. Another 
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possible explanation is that the degradation in performance 
of the automated agent was not significant enough to warrant 


concern. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSION 


Given the importance of automation to the future of 


military operations, it is vital that new technology and 





automated systems are designed and utilized in the most 





effective manner possible. In a resource constrained 





environment, these new systems that require humans and 
automation to work together must take into account the 


differences in how combined teams function as compared to 








human teams. These differences need to b researched and 


better understood in order to maximize overall performance 





of these new teams. 





The concept asserted by Sheridan (2002), that automated 
systems are designed based on automating the most basic 


functions, will not suffice in the future. Christoffersen 





and Woods (2002) suggest that the two critical 





characteristics that need to be present in a design from the 
beginning are observability and directability. The human 


users need to be able to see and understand what the 








automation is doing and intends to do, along with the 


ability to change the automation’s actions as necessary. 





Other suggestions include the need for better user 





interfaces and displays to increase coordination 


capabilities and the incorporation of better feedback 





channels to increase awareness and understanding. The key to 





ffective design and implementation of human-automated teams 





is to understand the nature of the differences between 





oy) 


human-human teams and human-automation teams and design to 


minimize their impact on performance. 





In the present study, 36 participants were organized 





into four-member teams (three participants and a 
confederate) to participate in a team performance 
experiment. Each team participated in two C3Fire scenarios, 


one with the confederate acting as a human player and the 
other with the confederate acting as an automated agent 
playing the same role. Both scenarios were necessary in 
order to compare the nature of the interactions across both 


types of teams. 





The original hypothesis, that the two team types would 
communicate differently, was only partially supported. The 
second hypothesis, that the human teams would have an 


overall higher team fficacy, was supported. The third 








hypothesis, that the trust among the team with the automated 








agent would be lower, was also supported. 





The results of this study apply directly to the HSI 





domains of manpower, personnel, and human factors 


engineering. Manning requirements are driven by system 





design, but just adding automation without evaluating the 


effect on the overall system performance will not lead to 





proper manpower decisions. Additionally, these new automated 





systems may be utilized to reduce manpower, but if not 
properly designed and implemented may lead to less 


productivity. The type of personnel recruited to work on 








these new teams will also undoubtedly change. Working with 








these new human-automated teams will require personnel with 


aptitudes and skill sets related to advanced technologies. 








In order for these new systems to be operated, maintained, 
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and supported, it will be necessary for designers to adjust 
system design while considering manpower and personnel 


implications. Additionally, the automation itself needs to 





b designed with consideration for the manpower and 








personnel who will be recruited to interact with the 


systems. 


The design of these new systems must also be guided by 


the appropriate human factors engineering principles that 





take into account the variability of human skills and 
abilities, the limits of human cognition, and manner in 
which trust is established and eroded. New types of 


interfaces, displays, and information sharing devices need 





to be designed and utilized to maximize team cognition. 





Effectiv feedback mechanisms that allow humans to 
understand and communicate with automation in an effortless 


fashion need to be incorporated into the design. 


B. RECOMMENDATIONS 


The empirical findings of this study warrant the 





consideration of designers, engineers, and acquisition 





professionals. This study suggests that system designers 


should consider the differences in communication, team 








efficacy, and trust between human-human and human-automation 
teams and work to improve the manner in which human —- 
automation teams perform. Appropriate communication 


protocols should be employed to facilitate efficient and 





complete transfer of information between all members of the 


team. The system must be designed so that the human users 





are aware of the automation’s current state and intended 








actions. In order to increas team fficacy, the human 


users need to be able to understand how the automation 
5d 


functions. The ‘hide everything in the black box approach’ 
must be abandoned; the abilities and shortcomings of the 


automation need to be transparent to the human users. 


With respect to trust, several design implications can 





be derived from this study. First, trust in automation is 


rapidly eroded when the automation does not perform as 








expected. One possible (but unrealistic) solution is to 





acquire automated systems that are completely reliable. A 
better solution is to provide users with cues that allow 


them to quickly and accurately calibrate their trust to the 





automation. Human team members need to know when to trust 











and when not to trust automated team members. Ignoring the 





performance differences due to team composition and 
continuing to design systems that do not account for such 
differences may have a profound impact on future military 


team capabilities. 


Cc. FOLLOW-ON RESEARCH 


Research in this area is still relatively new. There 





are many opportunities to advance our understanding of how 


to integrate automation and automated agents effectively 





into military teams. Further research is necessary to fully 





understand the differences in the interactions between 
members of human teams and human-automated teams. For 


example, one key improvement that could be made to the 





present study would be to standardiz th scenarios for 


duration. The scenarios need to be designed so that all 











teams would play for the same amount of time under all 





conditions. This could be accomplished by generating 


multiple fires throughout the scenario that are not capable 








of being fully extinguished before tim lapses. In order 
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to reduce large variability in team performance, additional 





training time should be allocated. Multiple C3Fire sessions 


prior to conducting actual data collection would lead to 


more consistent performance among teams. The final 


suggestion would be to conduct another study that would look 





at a proximity factor. To what extent does proximity affect 





trust and efficacy? If the participants never met face to 


face, would their trust and efficacy still be higher in the 





human-human condition than the human-automation condition? 


in the laboratory 


field 


Research should also be conducted 





with other simulations, as well as in settings. 





Research with actual autonomous agents and the actual user 
population needs to be conducted to ensure that the results 


valid beyond th xperimental 





of the present study are 





setting reported herein. 


Results of this study reveal compelling evidence that 


the addition of an automated agent to a human team changes 





the nature of communications amongst team members, impacts 


team efficacy, and alters the task distribution of the 








team. Statistical analysis showed that even though the 





total number of communications amongst both group types 


equivalent, 


human 


the type of communication was 


information was 


more 


different. 





shared to aid in decision-making, 


teams, 


nature. 


human teams, 


likely to be 


Whil 


th 





communications wer 





presenc 


mor 


of an aut 


likely to b 


while on human-automation 


directiv in 








tomated agent did not 





affect overall team performance, it did affect trust, 


efficacy, and task distribution of the team. Statistical 


analysis indicated that human team members had greater 








confidence in another human team member compared to the 
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automated agent. Analysis also showed that the human team 
members trusted the automated agent less, which directly 


affected task distribution. 


The results of this study suggest that there are 
important differences in the interactions of human - human 
teams compared to human-automation teams and demonstrate the 
need to consider the unintended impact of including 
automated agents as team members in military environments 
and other complex and dynamic systems. Proper design and 


implementation of future military systems will lead to new 





capabilities that will increase flexibility, adaptability, 


and overall mission effectiveness. 
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APPENDIX A. PRE TEAM EFFICACY QUESTIONNAIRE (TEQ) 


1. Default Section 


1. What is your assigned role on the team? 


O Command and Control 


O Fire Trucks 
O Gas Trucks 
O Water Trucks 


2. Please answer the following questions about the team. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


My role is clear in helping O O O O O 
to meet the team's 
objectives. 

I am confident in my 
ability to properly perform 
this role. 

I have confidence in the 
command and control 
operator's ability to 
properly perform the role. 
I have confidence in the 
fire truck operator's ability 
to properly perform the 
role. 

I have confidence in the 
gas truck operator's ability 
to properly perform the 
role. 

I have confidence in the 
water truck operator's 
ability to properly perform 
the role. 

All necessary roles on the 
team are being filled. 

The proper operators are 
filling the correct roles on 
this team. 

This team will be able to 
fulfill its responsibilities. 
This team will work 
together to get the job 
done. 


O 


O 
O 
O 
O 
O 
O 
O 
O 
O 
O 


OO 00 0 90 0 O 

COO OF} ORD ORO O 
OFO OF ORB ORO O 
OHO OF OFFO ORRO O 
OF OFF O8RBD ORO O 
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APPENDIX B. POST TEAM EFFICACY QUESTIONNAIRE (TEQ) 


1. Default Section 


1. What was your assigned role on the team? 


O Command and Control 


O Fire Trucks 
O Gas Trucks 
O Water Trucks 


2. Please answer the following questions about the team. 


Strongly Disagree Disagree Neutral Agree Strongly Agree 
My role was clear in O O O O O 
helping to meet the 
team's objectives. 
I was confident in my 
ability to properly perform 
this role. 
I had confidence in the 
command and control 
operator's ability to 
properly perform the role. 
I had confidence in the 
fire truck operator's ability 
to properly perform the 
role. 
I had confidence in the 
gas truck operator's ability 
to properly perform the 
role. 
I had confidence in the 
water truck operator's 
ability to properly perform 
the role, 
All necessary roles on the 
team were being filled. 
The proper operators were 
filling the correct roles on 
this team. 
This team was able to 
fulfill its responsibilities. 
This team worked 
together to get the job 
done. 


O 


OR} Of} O8RBOD ORO O 


OF OF O8RBDO ORO 


O 
O 
O 
O 
O 
O 
O 
O 
O 
O 


OPO OF O8RROD ORO O 
OO OF ORO OREO O 
OFO OF} ORR ORO O 
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APPENDIX C. PARTICIPANT COMMENTS FROM DEBRIEFINGS 





think the gas truck AI 





trucks. 
It was difficult to 
The 
In the first 








(automated) gas 


scenario 





to tell the gas 
they were doin 
inactive. I thi 
something and 
before the 





but 





gr 
nk 


think 


























Yes it was a little confusing wi 


type commands 
trucks screwed us. 


(human operator) 
command and control guy that 
trucks what to do if 


sucks. 


th the 








to the automati 














knew that was 
did not 1 





(automated) 


gas 


on. 


knew as the 


supposed 
ike what 


noticed one truck kept being 


it was G9, 
typed it in four or five 
thing finally moved. 
think 








so kept typing G 


9 refuel 
times 





So either 





doing it right or 


so maybe it was the quotation marks it was lookin 


did not trust 
myself initially. 











I (Command and Control 


(automated) 


Operator) 


finally put it in qu 





gas trucks so 


did not monitor t 











trucks, just the 


The a 


Communication was more difficult 


(automated) 


utomation needed more monitoring 


gas trucks. 





(with 








Sometimes h 
he did not, 





(th 
sometimes h 


automation) 





did what 





said, s 





did no 








12. 


work with. 
tro. tel] 
They weren’t doing what 











to be, 


Ss 


but when we actually had a person ther 











wasn’t 
otations 
g for. 





refueled 


he other 


that the human. 


the automation). 


ometimes 


med to do his own thing. 


t think the automated gas trucks were as easy to 
Towards the end it seemed like 
them what to do as if 


was having 
was a gas truck guy. 
thought they should do. 
the end they were not really going to wher 


Toward 
ded 





they n 





they wer 


going to where you would just guess they needed to go 
without me having to tell them. 








very much. 
commands. 

to say. 
say 








didn’t like working with the computer 
did not want to think about any special 
just wanted to be able to 
Like when you are working with 
‘hey get over there’. 








good job initially, 


(aut 


type what 





getting fuel to the fire trucks. 
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a human you just 
It seemed like it was doing 


but there towards the end, it wasn’t 


omation) 





wanted 


a 





13% 





16. 





19. 


20. 


21. 


22. 





thought among the people it was easier to coordinate 
than with the automation because we all generally had 
the idea to box the fire into that corner. 

















The chang (between the scenarios) was the Intelligent 
Agent and it sucked ass ‘yeah it sucked’ (background). 
























































The gas truck intelligent agent thing, well it sucks. It 
didn’t respond to the commands that I would send. I am 
the command and control and nothing that it is doing is 
making sense so I am typing furiously commands to it or 
then just trying to move the damn truck. 

am like refuel (to the automated agent) and then it 
doesn’t move so I try and move it and then it moves 
back. 

The automated, yeah that was terrible because I could 
tell when a person was doing it. 

Yes I did notice a difference in the two scenarios. In 
the second scenario (working with humans), we did not 





have as bad of time running out of fuel and water (as 















































compared to working with the automation). I think the 
collaboration was a lot better. They knew what I was 
going to do and they were able to react to that. 

Yeah it was definitely difficult to coordinate (with the 
automation). I was like, ‘what do I type?’ 

It was kinda hard to get my point across (with the 
automation) with the typing. I am used to voice chat. 

thought it was easier when the person moved the gas 

truck because the automation was overriding my commands. 




















I think the biggest thing that I noticed just from 
monitoring all the chat especially when you have an 
actual person as the gas truck I think in the chat box 
there needs to be some kind of line or something that 


says who this is actually from because you are getting 
all the commands from people and you don't know where 
they are coming from so you can't respond back that 











person without saying 'oh I assume Player B sent this so 
let me send it back to B'. You need some way to know who 
the chat is coming from. I thought it was definitely 








easier the second time (working with automation) when 
you don’t have a physical person sitting at a gas 
command and you don’t have another set of commands 
coming in, but that's just from lack of message traffic. 
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23%. 


24. 


20s 


But the ot 
the 


where 
trucks 
went 


so like 


trucks 
the 





in 


they didn't... 








station 
lik 


EO. fill wp, 





just 





station 
nothing 
just 





long that 


they refuel 


seemed like 
didn’t look 


It appeared that 


to a station to fill up, 
were out of gas and then instantly they were r 
don’t know if they went 
never even looked. 
d with their automated command a whole 




















or not, bu 
ever ran out 








said I do 








th 





t all 


of gas. 





ther thing is the umm. 
actually 
second 


were, 


scenario, li 





the 
ke 


wasn't 
(automated) 
if they actually 





looking at 
gas 





but it seemed like they 





But it 


(automated) 


fueling, 
to a gas 
seemed 








lot quicker than we could have refueled ourselves. It 
had no problems with 
to see if I 


gas. 


was out of gas or at a gas 


of the 


sudden 
(Other team member) 


everything... 
"Yeah 








quit checking the gas because 


quicker with the automated command. 











did not see any differenc 


betw 


n th 


The automated thing did a good job. 


it 





would not move 


supposed to. 


through 
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the 


£rre, 


was so full for so 
not even need to look at it." 


e gas trucks refueled a whole lot 


two scenarios. 
A couple of times 
but 


Mes 18. not 





TH 


IS PAGE 








NTENT 





ONALLY LEFT BLANK 
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